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ABSTRACT 

A distinction among evaluation, measurement and 
reports of terminal behavior is presented. For the purposes of 
evaluation, any response to a speech performance provides useful 
feedback to the communicator. In addition to the teacher's evaluation 
of the performance against "established standards", it is argued that 
a teacher's subjective responses should be combined with more 
extensive use of peer evaluation and self evaluation. All of these 
forms of feedback provide subjective but relevant sources of 
information about the communicative act. None of these should have 
any bearing on the teacher's accountability to the educational system 
or the student's course grades. With respect to measurement, the 
point is made that even if a valid and reliable measurement of speech 
performances were possible, it would be undesirable in terms of other 
course objectives ascribed to by most teachers. While refinement of 
measurement techniques should be continued for the improvement of 
written examinations and speech contests, the only measurement of 
classroom performances should be explicit and object purposes 

of behavioral outcomes, accountability for instruct ma> ua 
established by a description of the processes a student goes through, 
and course grades may be determined by reporting the student' s 
investment in the course in terms of the performances for which he 
receives credit. Performance contracting, process-concept grids, and 
other techniques are suggested as means for minimizing measurement 
and maximizing useful evaluation teacher accountability. 

(Author/CK) 
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Chen 1 was taking speech in high school, I was absent on the day that 
v/e did pantomines, and somehow talked the teacher out of making up the as- 
signment* It occurs to me that ray life has gone along quite well even with- 
out that experience. Yet I suppose that in many high school speech classes, 
whether it seems relevant or not, whether or not it is a totally humiliating 
experience, each student is still required to present a pantomine. 

In junior high school I won an essay contest on the topic "What made 
Abraham Lincoln Great?" I remember that the essay said that Lincoln was a 
great man because he was able to distinguish between the important and the 
essential* Evidently, my high school teacher had made the decision that 
doing a pantomine was not just valuable, not just important, but essential, 
because she required each student in the class to do 1:. 

As teachers of speech we are constantly challenged to make distinctions 
between the important and the essential* Each of us is involved in two con- 
tracts in which some of the terms are explicit and many others are assumed* 
We have one contract with the educational system* We make promises and com- 
mitments to society at large, legislators, boards of education, parents, 
principals and department heads* We also have contracts with our students* 
They demand at best a relevant and meaningful learning experience, at least 
a fair course grade for a reasonable investment of time and effort* All 
too often we find ourselves in conflict when the commitments to these two 
contracts seem inconsistent or just too overwhelming* I suspect that as 
a group we have failed in the past to demand all that is essential in our 
contracts with the educational system while our contracts with students 
have often insisted on too many things that are important but not essential* 
The systems of evaluating speech performances that are used in the 
1970* s will require us to focus on the essential elements in both of these 
contracts* Throughout the past decade I think that there have been two major 
educational trends that will have a strong influence on our evaluation prac- 
tices in the future. The first is the trend toward the specification of 
the outcomes of instruction* We are all familiar with the literature, the 
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legislation end the commissions that advocate, as Robert Majer would put it, 
"describing v/hat the student is doing when he is understanding group dis- 
cussion., " The second educational trend toward more student-centered teach- 
ing is equally familiar D We’ve read, often with a blush of recognition, 
the critiques of our schools that tell us that we are educating students 
’’from the neck up” in a setting that they view as a prison, using methods 
that they perceive as dehumanizing, teaching content and skills that they 
view as irrelevant. Many of us have been inspired by the descriptions of 
various alternative school and open classroom experiments. As we consider 
these two major trends in educational thought, v/hat kinds of speech class- 
rooms, with what kinds of teachers, using v/hat kind of evaluation systems 
do we envision? The single word that best describes the picture that comes 
to my mind is schizophrenic, 

X was privileged to hear Carl Rogers speak a few weeks ago on the topic 
of uniting ideas and feelings in learning. Following his speech a member 
of the audience asked, ’’What relationship exists between the kind of educa- 
tion you have just described a " the pre-planning of objectives?” Rogers 
answered, "Damned littlei" I suppose it is possible that there are teachers 
who are not bothered by the inconsistencies between these two trends. May- 
be you’re a behaviorist in the most mechanistic Skinnerian sense who be- 
lieves in deciding exactly how you would like all of your students to 
turn out and then reinforcing their behavior toward that end. Or maybe 
you’re the most far out kind of existentialist who walks into class and 
says, "Hi, I’m a person and you’re people. Let’s he human toge^'ier, ” T * r 
soj your role as an evaluator (or a non-evaluator ) is very clear. Actually, 
I don’t know many people who approach either of those caricatures. Most of 
us find much that is appealing in both of the educational trends that I 
have mentioned. And as a result, most of us get pretty confused, 

I’d like to outline the problems we face in three areas of evaluation 
behavior and then tell you why, for once, I’m more optimistic than Carl 
Rogers, , ^because I believe that we can solve these problems and combine 
the best of the world of pre-planned objectives with the best of the world 
of the open classroom. 

One kind of evaluation behavior that we’ll all be increasingly involved 
in is the description of the outcomes of instruction. The greatest reser- 
vation that I have about this essentially sound imperative is that it is 
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much easier to specify behavioral outcomes for the psychomotor domain and 
for the lower level skills in the cognitive domain than it is to describe the 
behaviors that accompany learnings in the higher level cognitive skills and 
in the affective domain* Cur discipline has traditionally played an inte- 
grative role among other academic disciplines and we have claimed that the 
concepts and skills that we teach are highly transferrable and generalizable® 
I think that we must be especially cautious in the next decade to maintain 
this essential emphasis in our contracting with educational systems® The 
other important, but less essential, specific skills and content areas must 
not come to be defined as the essence of our discipline merely because they 
are easy to specify and measure® We must be militant in ouir refusal to 
sell out to the kind of "cost accounting mentality " that urges us to de~ 
emphasize our essential objectives — analysis, synthesis, application and * 
evaluation skills and attitudes, values and feelings-- just because we cannot 
at the moment specifically describe and measure the manifestations of these 
learnings® 

A second kind of evaluation behavior is providing feedback to students 
about the effects of a communicative act® This feedback may be written or 
oral, verbal or non-verbal, immediate or delayed, subjective or objective, 
descriptive or prescriptive® We know from learning theory that knowledge 
of results is a critical reinforcer, and we know the 4 ’ mutvcat lop be- 

havior particular 11 ' the impact of e message on the receiver is 

essential to increasing comnunicative effectiveness* The major reservation 
that 1 have about this kind of evaluation behavior is that neitr-er teachers 
or students have been adequately trained in giving feedback in %ear\*3 that will 
sustain a growthful dialogue® We do not have a coherent body ci research 
from which to draw principles governing feedback behavior in the uiassroom 0 
My re" iew of the literature in this area revealed many theoretical writings 
e • n co3 and donuts of criticizing student speeches' . • *and may- studies 
in ] Moratory settings which related principles of learning theory to verbal 
behavior* But over a twenty year period I found fewer than a dcren empiri- 
cal stucies either describing or analyzing the effects of varietur methods of 
giving reedback in the speech classroom* ^ 
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Moreover, wo? have almost no knowledge of the emotional impact of speech 

feedback on individual students . Cur indirect and anecdotal evidence strong- 
ly suggests that even the most well-intentioned, constructively phrased 
speech feedback can have damaging emotional results. Gerald Phillips et al Q 
put it well when they say, "You can criticize a person’s tie and he can pass 
if off rather ligntly, but when you criticize a person's speech much more 
is felt. The whole personality is involved. 

A third kind of evaluation behavior we’re involved in is measurement, 
which I define as assigning numbers accordxng to rules. As early as the 
1940 ! s we had empirical evidence that speech teachers were not applying 
valid or reliable measures to the effectiveness of speech performances* 

Thirty years later, after extensive research into rating behavior, rating 
scale development and factor analytical studies of perceived communication 
effectiveness, where do we stand? In a paper presented at the Western 
Speech Communication Convention last month Larry Steward reviewed the re- 
search in this area and drew the following conclusions: 



* . o study after study indicates that we as teachers 
differ widely with each other, and even with ourselves 
over time# . .the thrust of the research is that, if 
we decided on a single system of evaluation, provided 
training for all of us in that single system, and then 
did not allow anyone to make evaluations until after 
several years of service, then we might achieve relia- 
ble results. . .1 must report that there is less room 
for optimism now, as to the discovery of an efficient 
method of evaluation, than there was in 1943* 






If measurement is assigning numbers according to rules, all of this 
evidence suggests that we are not even close to discovering those mysterious 
rules that govern our grading behavior. Yet teachers continue to assign nu- 
merical ratings or letter grades to speech performances Us though they were 
really measuring something. Robert Rostrom has this response to our grading 
practices: 



* . pit stretches the imagination to assume that J 
the result of the complicated inter-relationship we 
have built is going to be a five point interval scale 
called A B C D F which, by happy coincidence, is the 
same kind of judgment the registrar asks us to sub- 
mit each quartero 
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Even with rill the problems attending these three kinds of evaluation 
behavior, - describing instructional outcomes, providing feedback, and 
measuring effectiveness - I would like to offer two optimistic predictions 
about the kinds of evaluation systems that will emerge in speech classes over 
the next decade* First, I see our concept of evaluation and feedback 
becoming much broader* Instead of the teacher presenting an oral and written 
critique of formal speech performances and perhaps inviting a few class 
comments, the trend is toward immediate, on-going feedback of all communi- 
cation behavior, with a greatly increased emphasis on the interpersonal com- 
munication of small groups and dyads* I see the teacher’s role becoming much 
more authentic and spontaneous* Teachers are owning their responses instead 
of attributing them to textbooks or "good speakers"* They are acknowledging 
the subjectivity and fallibility of their responses* Instead of saying "That 
was a poorly organized speech," they are saying, "I got lost*" Instead of 
saying "Don’t interrupt*” they are saying, "I felt a little uncomfortable ~ 
minute ago because I wanted to listen to you but I wasn't sure if Jim was 
finished with what he was saying*" In such a system, an essential ingred- 
ient of effective feedback is present: reciprocity* The teacher’s own ccm- 

munication behavior is open to subjective, immediate student response* 

As the teacher's evaluations become less central and less sacred, there 
will be a concomitant increase in the role of peer evaluation and self 
evaluation* Among the peer evaluation techniques that are becoming more 
prominent are: more real class discussions of speech performances, more use 

of students as process observers, more use of audience surveys as means of 
assessing the effectiveness of informative and persuasive discourse, and 
finally more use of appropriate human relations exercises and "light eru 
counter" techniques that encourage students to discover and express their 
££ S ponses to each other as communicators* Among the self evaluation teoh~ 4 
niques that are gaining in popularity are: student journals of daily commu- 

nication experiences, self-analysis papers before and after speech perfor- 
mances -where the student describes his goals for the experience and his 
assessment of the outcome, and more use of audio and videotape recordings 
of classroom communication which the individual student may review in pri- 
vate and respond to according to his own needs* 
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Iv/ soccnr prediction is ti nt mea surer en t of speech performance will 
play a far less central role ir. the speech classroom. Most of us have said 
for years that we hate grading and that we wish we could grade on a pass/ 
fail basis* Recently more and more teachers have adopted that system, at 
least for the measurement of individual class assignments. I, for one, have 
graded my last speech performance! My only measurements of speech perfor- 
mances, essay exams and student papers and projects for the last few semes- 
ters have been the binary measurement of "credit" or "no credit yet." I 
am as explicit as possible about the conditions for receiving credit. If 
I can't specify and justify the conditions to the student's satisfaction, 

I omit them. I have been much more confident that I can make reasonably 
objective and reliable judgments under this system. I can s^ay, "Yes, you 
gave a speech that had a clear thesis sentence, and introduction, body and 
conclusion, used at least four different forms of support from at least three 
different sources" or "No, you didn't." I don't have those ridiculous con- 
versations with students where I hear myself lamely justifying why I gave 
a speech a C* instead of a B- while suspecting deep inside that C means 
Ook. , B means I liked it and A means I liked it a lot* 

Of course I still make judgments and express those judgments. But 

when I say, "I loved your introduction" or"I was really let down by your 

conclusion" or "I felt like I wanted to hear a few more examples of your 

second point," I f m responding and feeding back my response. I like myself 
better, and -thus probably function better as a teacher, when I admit that, in- 
stead of pretending that I'm measuring something. 

For the forseeable future, one other kind of measuremert will remain 
incumbent on speech teachers. We will probably have to continue to give 
course grades something like A, B, C, D or F to each student. The alter- 
native of allowing students to grade themselves on their perceptions of their 
own personal learning might be ideal, but for many of us our contract 
with the educational system binds us, in conscience or in fact, to guar- 
antee some sort of correlation between course grades and ourf stated 
objectives for the course. There is no reason, however, that course grades 
must reflect the distinction between meeting our objectives, meeting them 
very well and meeting them fantastically well. Criterion measures in ob- 
jectives can be binary and grades above C can express that students have met 
additional objectives (of ours or their own) or that they have chosen to 
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meet sene course objectives through doing more projects than were required* 

Rod Hart describes his system of giving course grades in this way: 

Whichever system I decide upon, it always is designed 
to record the amount of investment a student decides 
to make in the course* A kid does as much or as lit- 
tle as he w§mts and his grade is a function of that 
investment. 

In short, I think that some sort of contract based on the quantity of 
work for which credit was received would best allow us to honor both our 
educational contracts. The teacher keeps seme element of prescription over 
the course experience and some level of quality control, yet every student 
of reasonable intelligence and ability is allowed to attain any grade that he 
is willing to commit himself tc work for. 

Time does not permit me to elaborate on the specific systems that are 
being used or might be used to implement this approach. Basically, there are 
three steps that a teacher goes through. 

1. Establishing course objectives. Here is where some real soul 
searching comes in as we try to distinguish between the important and the 
essential. Will the world or the student really suffer if you omit: The 

student will understand the principles of parliamentary procedure? If your 
best professional judgment says yes, then include it. Otherwise, as Peg 
Bracken «s cookbook says about leftover food, "When in doubt, throw it out." 

I predict: that you will find yourself writing more course objectives like: 

The student will understand the role of audience analysis in a variety of 
communicative forms and settings, including at least two of the following: 
persuasive speaking before a student audience, persuasive speaking before a 
community audienge. a debate, a ■pars&fl. discussion, a formal business meeting, 
a dramatic production, an oral interpretation program, etc. 

2. The second step, after the establishment of objectives, is for 
the teacher to generate (and describe and weight and provide resources for) 
several communi cation experiences which might accomplish each course object- 
ive. For instance, a student might meet the objective of understanding 
communication models by reading chapter seven and answering the study ques- 
tions at the end, passing an objective test, writing a book report on any of 
several books you could specify, presenting a symposium discussion on models, 
or designing his own communication model. If you feel that these activities 
involve vastly different amounts of time or educational value, then you will 
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have to go to some sort of point: system where credit for one might count 5 
points and credit for another counts 15 points. 

3. This leads us to the third step. The teacher establishes a contract 
for course grades that is clear and understandable to the student. One very 
simple model that each person here is familiar with is the scout handbook 
system. I remember that when I got the outdoor cooking badge, every girl in 
the troop had to make a tin can stove, but some of us made those awful things 
out of graham crackers and marshmallows and Hershey Bars and some other airls 
were making pancakes and some of the girls were making cakes in paper bags. 
Somebody older and wiser had decided that we had to do the projects with the 
stars beside them, but I felt pretty important picking out which other pro- 
jects I wanted to do. Sometimes I even did extra onesi So the simplest course 
contract says: for a C do the starred activities and any three others, for a 

B any six others, for an A any 9 others. More complex contracts use point 
systems: For a C you must earn credit for activities totalling 100 points in- 

cluding the two takehome exams and at least three speech performances including 
the informative speech and the persuasive speech. For a B you must receive 
credit for activities totalling 150 points, etc. 

A final syctem, called the process-concept grid (which I borrowed from 
my sister's fourth grade science class) is even more complex. On the verti- 
cal dimension of the grid you list the processes that you want the student 
to experience. These might be drawn from one of the taxonomies such as: com-' 
prehension, analysis-synthesis, application, and evaluation. Or you might 
develop your own list such as: listening critically, doing research, organi- 

zing material, delivering speeches, etc. On the horizontal dimension you would 
list concepts of content areas such as argumentation, group dynamics, commu- 
nication theory, mass media, etc. Then you generate activities that fit each 
square. The contract reads something , like: for a C receive credit for any 

ten activities including at least one activity from each of the starred 
squares, etc. 

These new approaches to grading create all kinds of new problems in 
scheduling and record keeping and they require a tremendous amount of time to 
establish since nearly every activity will require a separate and detailed 
handout and certain resource materials. There are some creative ways of 
dealing with these problems that various teachers have devised. Scheduling 
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can be turned over to the students to work out as a group task* Brian Hollern 
handled the problem of sizudent procrastination in submitting the projects by 
giving bonus points for activities completed before a certain date* 

I am optimistic about the systems of evaluation that I see emerging in 
the 1970 r s* Teachers and researchers are responding responsibly to the de- 
mands for the specification of the outcomes of instruction without compromising 

7 

their professional and personal integrity* The scope of evaluation is being 
broadened and the inherently subjective nature of many of our responses to the 
conplex communicative act is being acknowledged* While the measurement of 
speech effectiveness must continue to be of concern to theory builders, class- 
room teachers are limiting their measurement behavior to judgments that they 
can make with reasonable reliability* Most important, the entire system is 
becoming more valido Students will stop working to psyche out the elusive 
image that each of us has as the M great communicator" and start to deal with 
more realistic and relevant communication problems* The systems that are 
emerging are based on more authentic, role- free student teacher relationships* 
They acknowledge the individual differences in students r life goals and 
learning styles and require the student to take more responsibility for the 
planning and evaluating of his own educational experiences* Teachers are .4 
struggling with the difficult distinctions between the important aid the essen- 
tial and are developing contracts that represent good faith both with educa- 
tional systems and with students* 




10 



References 



MoDo Albright, "The Response of Students in Fundamentals of Speech 
to Oral Criticism," unpublished masters thesis, Northern Illinois 
University, 1967; D 0 H 0 Arnold, "Perception, Recall and Influence 
of Criticism on Students in Fundamentals of Speech, unpublished 
masters thesis, Illinois State University at Normal, 1964; R*N* 
Bostrum, "The Effect of Speech Attitudes of Positive and Negative 
Evaluation Comments on Classroom Speaking, unpublished doctoral 
dissertation, State University of Iowa, 1961; L„ Freyman, "A Sur- 
vey to Discover Instructional Procedures in Use Following Student 
Speech Performances, unpublished doctoral dissertation, Western 
Reserve University, 1954; D.M® Hinde, "An Investigation into the 
Effectiveness of Criticism of Speech Delivery When Given Before, 
During and After Speech Performance, unpublished masters thesis, 
State University of Iowa, 1949; J*M 0 Lutz "Students 1 Reactions 
to Criticism in the High School Speech Class," unpublished masters 
thesis, Pennsylvania State University, 1967; JoAo Sprague, "An 
Investigation of the Written Critique Behavior of College Commu- 
nication Instructors," unpublished doctoral dissertation, Purdue 
University, 1971; J* Timmons and LoR* Osborne, "Grading Practices 
in College Fundamentals Courses," Kans as Speech Journal , (December 
1964) pp e 24-33; I 0 D 0 Warren, "The 'Effect of Three Forms of Critic 
cism on Volume," unpublished masters thesis, Northern Illinois 
University, 1966; One study outside the field of speech is of 
special note: E.Bo Page, "Teacher Comments and Student Performance: 

A Seventy -four Classroom Experiment in School Motivation," Journal 
of Educ ational Psychology , (August, 1958), pp*l73-181o 

2 . 

GoMo Phillips, R.Eo Dunham, R 0 Brubaker, and D c Butt, The Develop - 
ment of Oral Communication in the Classroom (New York, 1970) p,21 



LoAo Steward, "A Critique of Techniques of Evaluation in the Speech 
Classroom," a paper presented at the convention of the Western 
Speech Communication Association in Fresno, California, November 24, 
1971* 

4 

RSI?, Bastrtam, "The Problem of Grading', " Sneech Teacher, XXII 
(’November, 1968), pp. 287-292* 



O 

ERIC 



11 



RoP* Hart, "On Teaching 114: Reflections of Paranoia from Someone 
Unencumbered by Vast Experience,” unpublished paper, Purdue Uni- 
versity, 1970. 



6 3.P, Holleren, M An A-Group Approach to Advanced Public Speaking,” 
Speech Teacher , XX (September, 1971), pp. 216-219. 



7 

See for example, the First Annual Report of the Committee on 
Criteria- Based Instruction .of the California High School Speech 
Association. 




12 



Author Abstract 



Evaluation: Problems in Evaluating Speech Communication Performance , Jo Sprague, San Jose State 
College. 

The practice of making teachers accountable for specific outcomes of instruction will have a positive 
impact on speech education only if systems of accountabih t negotiated and classroom evaluation 
systems are established with an awareness of other important u iucationa« trends, student demands for 
''relevance", the individualization of instruction, more role-free student-teacher relationships, etc. 

This paper distinguishes between evaluation, measurement and reports of terminal behavior. 

1. Evaluation. Any response to a speech performance provides useful feedback to the communicator. 
In addition to the teacher's evaluation of the performance against "established standards , it is argued 
that a teacher's subjective responses should be combined with more extensive use of peer evaluation and 
self evaluation. All of these forms of feedback provide subjective but relevant sources of information 
about the communicative act. None of these should have any bearing on the teacher's accountability to 
the educational system or the student's course grades. 

2. Measurement. Even if a valid and reliable measurement of speech performances were possible, it 
would be undesirable in terms of other course objectives ascribed to by most teachers. While refinement 
of measurement techniques should be continued for the improvement of written examinations and speech 
contests, the only measurement of classroom performances should be a binary credit or no credit 
judgment. Criteria for receiving credit for a performance should be explicit and objective. 

3. Behavioral Outcomes. Accountability for instruction may be established by a description of the 
processes a student goes through, and course grades may be determined by reporting the student's 
investment in the course in terms of the performances for which he receives credit. Performance 
contracting, process-concept grids, the Scout Handbook methods, and other techniques are suggested as 
means for minimizing measurement and maximizing useful evaluation, teacher accountability and fairness 
in course grades. 
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